Complexity of applications and computer programs continues to increase as users continue to expect more functions from smaller and smaller devices. In order to meet this demand, many products now include multiple ways to process information, for example using multiple processing units.
In order to meet the demands of computer users, designers have started developing ways in which multiple processing units, either multiple processing units on a single silicon die or multiple processing units in communication, can be integrated to collectively handle multiple tasks required for an application or program to run. For example, one processing unit may be handling graphical rendering for a computer game, while another may be handling processing the users actions, and another may be handling communication with other users' computers.
Integrating multiple processing units to simultaneously handle different tasks can be a very difficult task. The designers of these systems must make sure that all the tasks are completed, but that no two processing units are attempting to complete the same task. Thus, in current computer systems, a central or master processor first divides up the tasks into discrete tasks and then assigns the tasks to the different processing units, such as APUs, GPUs, ASICs, etc.
Dividing up the tasks ahead of time allows for the system to process multiple different tasks at the same time, while also guaranteeing that no task is processed twice. But this also leads to a less than optimal solution. If for example, one of the processing units finishes up the tasks assigned to the processing unit before the others have started all of their tasks, the processing unit remains idle, even if there is additional work to be accomplished that has been assigned to other processing units. Designers often try to address this issue preemptively by estimating how long each task will take on each processing unit, and determining a theoretical optimal solution. But, if a processing unit does not execute exactly how the processing unit has been modeled, this still leads to processing units being idle while there is more work to be done.